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Application No.: 09/679,391 
Filed: October 3, 2000 
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Group Art Unit: 2676 
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APPLICANTS' BRIEF ON APPEAL 

Mail Stop A ppeal Brief-Patents 
Commissio] <er for Patents 
P.O.Box 1450 
Alexandria, VA 22313-1450 

I. REA L PARTY IN INTEREST 

The .neal party in interest is Creatier Interactive, IXC by virtue of an Assignment from the 
U.S. Bankn.ptcy Court to Creatier Interactive LLC, recorded on Reel/Frame 015036/0672. 
H. R£J ATED APPEALS AND INTERFERENCES 

Thai are no other appeals or interferences known to the Appellants or the Appellants' 
representative, which are believed to directly affect or be directly affected by or have a bearing 
on the Boar. I's decision in the pending appeal. 
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in. status of claims OCT 1 1 2006 

Claiais 31-38 are pending. The final rejection of claims 31-38 form the basis for this 
appeal- Claims 31-36 have been rejected under 35 U.S.C. §102(e) as being anticipated by 
Rangan et ; J., U,S. patent no. 6,198,833, Claims 37 and 38 have been rejected un 35 U.S.C. 
§ 103(a) as being unpatentable over Rangan et ai, U.S. patent no. 6,198,833 in view of Issadore- 
Banreca et aL, U.S. Patent No. 6,205,231. 
JV. STATUS OF AMENDMENTS 

All -intendments have been entered. The claims as currently amended are attached as an 
Appendix. 

V. SUI\ tMARY OF CLAIMED SUBJECT MATTER 

The present invention relates to an automatic authoring system for interactive television. 
Jn such intci-active television systems, it is desirable to **ta|T items of interest that may appear in 
the video fiames which may be of interest to a viewing audience. Such interactive television 
systems all^w the user to click on the tagged item during the broadcast for additional information 
on the item of interest* In order to provide such capability, items of interest have to be tagged in 
all video frames where the items of interest may appear. In a NTSC broadcast, video frames are 
broadcasted at a rate of about 30 frames per second. Thus, a one-hour broadcast would involve 
108,000 video frames. Tagging the 108,000 video frames for the various items of interest is an 
arduous task, to say the least- In order to solve this problem, automatic authoring systems have 
been developed. These systems allow a user to mg items of interest in a single video frame and 
die system uutomatically locates and tags these items in succeeding video frames. 

The present invention relates to a relatively simple method and apparatus for tagging 
items of imerest in video frames by associating a color 120 of an object with an event. In 
particular* me system determines the locations within each video frame where a selected item 
may appeal The system determines the color value of the selected item by its digital color 
mathematic properties. For example, one embodiment of the invention utilizes the RGB 
standard widch separates the red 220, green 230, and blue 240 properties as the color as 
numerical values 250, Another embodiment uses the HSV characteristics of the color which 
define the hae 320, saturation 330 and value 340 and components of color. 

One . an item of interest is determined in an initial frame, the system then automatically 
locates the > elected item based on the color mathematical characteristics in the remaining frames 
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in the broadcast. These items are then tagged and Jinked to an event, such as a URL. One 
embodiment of the invention uses an edge detection scheme to define the boundaries for the 
colors of uV selected object. An important aspect of the present invention is that the invention 
utilizes a relatively simple process for otherwise completing a rather arduous task. 

Claims 31-36 recite, in combination, methods and systems which include which require 
determining the location where an action by a pointing device has occurred (See Fig. 1 and page 
15, lines 10 17) and deterniining a color value or color range of the selected location (See Fig. 1, 
Page 15, \inss 18-20) and automatically associating an event with the color value in succeeding 
video frame ; (See page 15, lines 19 and 20) Claims 37 and 38 are similar but relate to an edge of 
a selected ol-ject.( See page 24, line 13 to page 2S, line 9) 
VI. GR( iUNDS OF REJECTION TO BE BE VIEWED ON APPEAL 

I. Claims 3 1-36 stand rejected as being anticipated by Rangan et al., U.S. patent no. 
6,198,833 (hereinafter referred to as "Rangan.") 

II. Claims 37 and 38 stand rejected under 35 USC 103(a) as being unpatentable over 
Rangan et al., U.S. patent no. 6,198,833 in view of Jssadore-Barreca et al., U.S. Patent No. 
6,205,23 1 (hereinafter referred to as "Issadore-Barreca.") 

VU. ARGUMENT 

I. REJECTION OF CLAIMS 3 1-36 UNDER 35 U.S.C. $102(E1 AS BEJNC 
UNPATENTABLE OVER RANGAN Ey AL. 

It is i espectfully submitted that the claims 3 1 -36 stand or fall together. Claims 3 1 and 32 

have been rejected under 35 U.S.C. § 102(e) as being anticipated by Rangan, et al., U.S. Patent 
No. 6,198,8:^3. In order for there to be anticipation, each and every one of the elements of the 
claims must t>e found in a single reference. It is respectfully submitted that Claims 31 and 32, as 
well as Claims 33-36, include elements clearly not suggested or disclosed in the Rangan, et al. 
patent. For example, Claim 31 recites, in combination, a method which includes a step for 
deterroinwg ihe location where an action by a pointing device has occurred- This claim also 
recites the siep of determining a color value with the selected location and automatically 
associating aa event with the color value in succeeding video frames. Claim 32 is similar. 



PAGE 7/16 * RCVD AT Id/1112006 4:12:38 PM [Eastern Daylight Time] ' SVR:USPTO-EFXRF-5/20 • DNIS:2738300 * CSID:3129021061 * DURATION (mm-ss):02-16 



10-11-06 03:14pm From-KATTEN MUCH IN ROSENMAN 13129021061 3129021061 T-660 P. 06/16 F-581 



Claim 35 recites a system in which the locations of an object selected by a pointing device are 
determined. This claim also recites detennining a color range for the selected object and 
associating an event with the color range in succeeding video frames. Claim 34 is similar. 
Claims 35 a.id 36 are similar but relate to a color partem. Finally, Claims 37 and 38 are similar 
but relate to an edge of a selected object. The Rangan patent does not disclose such elements. 

&anj an, on the other hand, tracks a rectangular wire frame, having a fixed size around the 
location wb re the pointing device was exercised by comparing the color of individual pixels in 
the fixed wi. e frame from one frame to the next. Rangan does not define an object based on its 
color, nor di es it dynamically track the object based on color as in the claims at issue. Rangan 
tracks objeos based on the movement of individual pixels from frame to frame within a wire 
frame. This movement tracking is based on the initial wire frame location which is tracked in 
each frame and does not automatically associate a color with an object as in the claims at issue. 
Tracking movement of the wire frame in Rangan demands testing of individual coordinates 
against their counterpart from the previous frame. This testing requires significant data to be 
computed at each frame. Further, it is respectfully submitted that Rangan's color signature are 
not representative of a continuous color pattern, m particular, Rangan's color signature records 
the color valae of every pixel in a fixed area rather than detecting a color pattern based on a 
region of interest. As such, it should be clear that claims 31-36 disclose elements clearly not 
disclosed or suggested by Rangan. Thus, the Board is respectfully requested to reverse this 
rejection. 
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°- REJECTION OF CLAIMS 3 7-38 UNDER 35 U.S.C. S1»3(A) AS BEING 
UNPATENT ABLE OVER HANGAN ET AL. IN VIEW OF 

issadqre-barreca rt Ar . 

Claims 37 and 38 recite, in combination, a method and system for automatically 
identifying 4 bounded object based on color characteristics in a series of video frames and 
associating m event with said object. It is respectfully submitted that claims 37 and 38 are not 
suggested by Rangan or Jssadore-Barreca. As discussed above, Rangan does not track objects in 
the same ra;.imer as the system in the current claims. In particular, Rangan tracks the movement 
of an object of fixed shape by comparing the color of indivifjual pixels in a fixed matrix from one 
frame to die next. Nowhere does Rangan disclose or suggest tracking an object based solely on 
its color, noi does it dynamically change the shape and size of the object based on color. 

The issadore-Barreca, et al. patent was cited for disclosing an edge detection scheme. 
However, tb s Examiner has failed to show that an edge detection scheme would be compatible 
with the fire. 1 wire frame tracking method taught by Rangan. Indeed, both systems would appear 
to be incorap anble with each other. 

Also, it is respectfully submitted that the Examiner has failed to establish a prima facie 
case of obviousness as set forth in MPEP § 2142 and § 2143. In order to establish a prima facie 
case of obvitusuess, three criteria must be met as set forth in MPEP § 2143. 

*Firsi, there must be some suggestion or motivation, either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the 
art, to modify the reference or to combine reference teachings. Second, there 
must be a reasonable expectation of success. Finally, the prior an reference (or 
references when combined) must teach or suggest all the claim limitations. 

The leaching or suggestion to make tfie claimed combination reasonable 
expectation of success roust both be found in the prior an, not in the Applicant's 
disclosure." 

It is respectt,41y submitted that the examiner is combining two references in support of the 
rejections without a showing that the motivation for the combination was suggested by the 
reference itself. Moreover, none of the references, either singly or in combination, teach all of 
the elements of the claims. For example, all of the claims recite automatically associating an 
object with it color in a plurality of video frames. None of the references cited disclose such an 
automatic aumoring system. Moreover, there is no reasonable expectation that the combined 
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references would succeed in an interactive TV application. For all of the above reasons, the 
Board is reS|>ecrfully requested to reverse the Examiner's rejection of claims 37 and 38. 
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CONCLUSION 



OCT 1 1 2006 



The Board is respectfully requested to reverse the rejection of the claims by the 
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CLAIMS ON APPEAL 

31. A method for automatically identifying an object in a plurality of video frames 
and associai mg the object with an event comprising: 

detej mining a location in one of said video frames where an action by a pointing device 
has occurreii defining a selected location; 

detei mining a color value of said selected location; and 

automatically associating an event with said color value of said selected location in said 
one video fi4me and automatically associating events with said color value in succeeding video 
frames. 

32. a system for automatically identifying an object in a series of video frames and 
associating *.n event with said object, the system comprising: 

a system for determining a location in one video frame where an action by a pointing 
device has occurred defining a selected location; 

a system for determining a color value associated with said selected location in said one 
video frame; and 

a sysiem for automatically associating an event with said color value in succeeding video 

frames. 

33. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates of an object in a video frame that is selected by a pointing 
device defining a selected object; 

determining a color range for said selected object; and 
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automatically associating an event with said color range of said selected location in said 
one video fi ame and automatically associating events with said color value in succeeding video 
frames. 

34. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising; 

a system for determining the coordinates of an object in one video frame that bos been 
selected by u pointing device defining a selected object; 

a system for determining a color range for said selected object in said one video frame; 

and 

a system for automatically associating an event with said color range in said one video 
frame and succeeding video frames. 

35. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates of an object selected in one of said video frames by a 
pointing device defining a selected object; 

deten oinmg a color pattern for said selected object; and 

auton .atically associating an event wuh said color pattern of said selected location in said 
one video frame and automatically associating events with said color pattern in succeeding video 
frames. 

36. A system for automatically identifying an object in a series of video frames and 
associating ai* event with said object, the system comprising: 

a system for determining the coordinates of an object in one video frame selected a 
pointing device defining a selected object; 
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a system for determining a color pattern for said selected object in said one video frame; 

and 

a system for automatically associating an event with said color pattern in said one video 
frame and succeeding video frames, 

37. A method for automatically identifying an object in a plurality of video frames 
and associat»ng the object with an event comprising: 

detei raining the coordinates along one edge of an object in one of said video frames 

selected by 4 pointing device defining a selected object; 

dete? mining the color values along said edge of said selected object; and 

automatically associating an event with said color values along said edge in said one 

video frame and automatically associating events with said color values in succeeding video 

frames assot iated with the color values. 

38. A system for automatically identifying an object in a series of video frames and 
associating ;<n event with said object, the system comprising: 

a system for determining the coordinates along one edge of an object in one video frame 
selected by *i pointing device defining a selected object; 

a system for determining the color values along said edge of said selected object in said 
one video fr.one; and 

a system for automatically associating an event with said color values in said one video 
frame and succeeding video frames. 
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